The present invention relates to systems for recognizing and tracking occupants in fixed environments under variable illumination conditions. In particular, the method disclosed herein is designed for use in sensing and tracking automobile occupants for xe2x80x9csmartxe2x80x9d airbag systems. Generally, however, the method may be adapted to a wide variety of uses involving motion sensing and tracking, such as gesture recognition for controlling and interacting with computers, computer gaming involving control via gestures and body positioning, xe2x80x9csmartxe2x80x9d home applications such as identification and tracking of occupants, and security systems to determine the identity of an occupant, among others.
Conventional vehicle occupant safety systems typically utilize at least one mechanical, electromechanical, or electronic acceleration sensor fixed to the vehicle in order to sense acceleration. The sensor output is fed to a discrimination circuit for comparison with a predetermined threshold value. If the threshold value is exceeded, the discrimination circuit will send a signal to activate an occupant safety restraint device, such as an air bag or passive seat belt mechanism.
Conventional automotive crash detection systems based solely on accelerometer signals are not designed to account for variations in occupant type or position in determining whether to actuate a safety restraint device. Rather, these systems typically assume the existence of a nominal condition, such as the presence of an average male occupant without a seat belt. This assumption is designed to ensure proper actuation of the safety restraint device over a large range of possible occupant configurations when abrupt deceleration of the vehicle is detected by the accelerometer. Such an assumption, however, results in undesired actuation of the safety restraint device in over a wide variety of conditions, including for example, marginal crash situations where seat belt protection alone is sufficient, situations when the occupant is positioned such that actuation of the safety restraint device could cause injury, situations when the occupant type is inappropriate to warrant deployment of the safety restraint device (such as the case where the occupant is a rear-facing child seat), and in situations where no occupant is present.
Accordingly, there is clearly a need for an occupant sensing system that can tailor the actuation of a safety restraint device in a way that accounts for varying occupant conditions within a vehicle. For example, in cases where airbags are used as the safety restraint device, when an occupant is leaning forward toward the instrument panel there is a high risk of injury due to the force generated during airbag inflation. Even if the occupant is not too far forward at the beginning of the collision, the occupant will generally move rapidly forward relative to the interior of the vehicle. The occupant""s head will tend to lead the occupant""s body during this motion, and the occupant will move into the airbag deployment space as the vehicle rapidly decelerates. In this situation, the occupant will likely be injured by the impact of the inflating airbag. There are a number of factors that influence the forward motion of the occupant in addition to the initial position. These factors may vary markedly from one occupant to another, and include such variables as occupant size and type, whether the occupant has secured a seat lap belt and/or shoulder harness, whether and to what extent pre-impact braking (which tends to move the occupant forward relative to the interior) has occurred, and whether the occupant tenses during the collision. Also, variations in the crash intensity itself will cause considerable variation in occupant acceleration. Thus, there is a need for a system that provides reliable detection of the occupant type and position for analysis in making the decision whether, and how, to deploy an airbag.
When determining the occupant type, it is first important to determine whether the seat is occupied at all. Then, if it is occupied it is very important to classify the occupant in order to differentiate between various types of occupants such as large and small adults, children, and various child seats. Such information may be used to prevent deployment in undesirable situations, such as with a rear-facing child seat (RFCS) or when the seat is unoccupied. It can also be used to adjust the strength of the airbag deployment in accordance with the occupant""s size and/or type. Any means for determining the type and position of an occupant must be highly reliable to properly tailor airbag deployment. In the case of a child seat, achieving reliability is especially difficult because there are over thirty-five different infant seat configurations available. Additional difficulty is created by the fact that car seats and other features of an automobile interior are adjustable. An effective occupant sensing system must be capable of providing reliable data sufficient for operation of both front and side deploying airbags.
Since conventional crash discrimination systems cannot detect the wide variety of occupant types and positions present in automobile crashes, they have not proven to be completely satisfactory. In response, the prior art has attempted to overcome these deficiencies by providing arrangements that are generally directed at detecting occupant presence, size, or position relative to some fixed structure in the vehicle. Following are a few examples of such prior art arrangements.
U.S. Pat. No. 5,398,185 to Omura discloses a system for optimizing deployment of occupant restraint devices which utilizes a combination of a plurality of seat sensors, a card reader for inputting data regarding the physical characteristics of the occupant, and two telecameras to compute a value characteristic of each interior vehicle element and the occupant""s estimated behavior relative thereto.
U.S. Pat. No. 5,802,479 to Kithil provides a roof-mounted occupant position sensor array of capacitive coupling occupant position sensors, to determine position and motion of a occupant by analysis of distances of the occupant to the various sensors of the array and by analysis of the changes of these distances over time.
WO 98/48372 Published International Application to Farmer et al. discloses a method and system for detecting vehicle occupant type and position utilizing a single camera unit positioned, for example at the driver or occupant side A-pillar, to generate image data of the front seating area of the vehicle. The invention makes use of a 2-D grid in order to detect which regions of the grid are occupied and to compare the profile generated with stored profile data for a variety of occupant types to distinguish between occupant types such as forward or rear facing child seats, and occupants.
While these arrangements have provided an improvement over conventional crash discrimination systems, there still exists a need for a system which can tailor air bag deployment based on the specific type of occupant present in the vehicle, rather than utilizing broad categories based simply on occupant size. Furthermore, with the widespread use of air bags in motor vehicles, it has become apparent that with certain types of occupants, such as RFCS, airbags have a potential of causing more harm to the occupant than the crash itself. Therefore, deployment based on size alone is insufficient, and occupant type must also be specifically considered.
The prior art has attempted to distinguish infant seats from other occupants by using conventional distance measuring techniques to detect the amount and extent of possible occupant movement and by using weight-sensing arrangements to detect passenger weight. In either arrangement, threshold values are used to classify an object as either an occupant or an inanimate object. However, simply using weight sensors and/or movement monitoring has provided neither the level of discrimination needed between occupant types nor the reliability necessary to achieve xe2x80x9csmartxe2x80x9d control over air bag deployment. As a result, a need still exists for a system that can automatically and continually determine occupant type and position in a reliable and cost-effective manner.
The present invention is directed to an apparatus and method that satisfies the need for a system that can automatically and continually determine occupant type and position in a reliable and cost-effective manner. A computer vision system for recognizing and tracking occupants in a fixed space under variable illumination conditions includes at least one sensor fixedly positioned to capture light reflected from a fixed space just prior to the entrance of an occupant to form an initial image of the unoccupied fixed space. After entrance of the occupant, the sensor continues to form images of the occupied fixed space at a desired sampling rate. The initial image and the subsequent images are compared in a processing subsystem to dynamically determine occupant type and position. Optionally, the system may include an artificial light source to aid processing during times when natural light is insufficient. The present invention may be modified via sampling rate, image resolution, and image partitioning to provide an accurate and reliable sensing system for applications including xe2x80x9csmartxe2x80x9d airbags, computer gaming, adaptive automotive controls, and xe2x80x9csmartxe2x80x9d homes, among others.